A normal ear transmits sounds as shown in FIG. 1 through the outer ear 101 to the tympanic membrane 102, which moves the bones of the middle ear 103 (malleus, incus, and stapes) that vibrate the oval window and round window openings of the cochlea 104. The cochlea 104 is a long narrow duct wound spirally about its axis for approximately two and a half turns. It includes an upper channel known as the scala vestibuli and a lower channel known as the scala tympani, which are connected by the cochlear duct. The cochlea 104 forms an upright spiraling cone with a center called the modiolar where the spiral ganglion cells of the acoustic nerve 113 reside. In response to received sounds transmitted by the middle ear 103, the fluid-filled cochlea 104 functions as a transducer to generate electric pulses which are transmitted to the cochlear nerve 113, and ultimately to the brain.
Hearing is impaired when there are problems in the ability to transduce external sounds into meaningful action potentials along the neural substrate of the cochlea 104. To improve impaired hearing, hearing prostheses have been developed. For example, when the impairment is related to operation of the middle ear 103, a conventional hearing aid may be used to provide mechanical stimulation to the auditory system in the form of amplified sound. Or when the impairment is associated with the cochlea 104, a cochlear implant with an implanted stimulation electrode can electrically stimulate auditory nerve tissue with small currents delivered by multiple stimulation contacts distributed along the electrode.
FIG. 1 also shows some components of a typical cochlear implant system, including an external microphone that provides an audio signal input to an external signal processor 111 where various signal processing schemes can be implemented. The processed signal is then converted into a digital data format, such as a sequence of data frames, for transmission into the implant 108. Besides receiving the processed audio information, the implant 108 also performs additional signal processing such as error correction, pulse formation, etc., and produces a stimulation pattern (based on the extracted audio information) that is sent through an electrode lead 109 to an implanted electrode array 110.
Typically, the electrode array 110 includes multiple stimulation contacts 112 on its surface that provide selective stimulation of the cochlea 104. Depending on context, the stimulation contacts 112 are also referred to as electrode channels. In cochlear implants today, a relatively small number of electrode channels are each associated with relatively broad frequency bands, with each stimulation contact 112 addressing a group of neurons with an electric stimulation pulse having a charge that is derived from the instantaneous amplitude of the signal envelope within that frequency band.
In some coding strategies, stimulation pulses are applied at a constant rate across all electrode channels, whereas in other coding strategies, stimulation pulses are applied at a channel-specific rate. Various specific signal processing schemes can be implemented to produce the electrical stimulation signals. Signal processing approaches that are well-known in the field of cochlear implants include continuous interleaved sampling (CIS), channel specific sampling sequences (CSSS) (as described in U.S. Pat. No. 6,348,070, incorporated herein by reference), spectral peak (SPEAK), and compressed analog (CA) processing.
FIG. 2 shows the major functional blocks in a typical cochlear implant signal processing system wherein band pass signals are processed and coding to generate electrode stimulation signals to stimulation electrodes in an implanted cochlear implant electrode array. For example, commercially available Digital Signal Processors (DSP) can be used to perform speech processing according to a 12-channel CIS approach. The initial acoustic audio signal input is produced by one or more sensing microphones, which may be omnidirectional and/or directional. Preprocessor Filter Bank 201 pre-processes the initial acoustic audio signal with a bank of multiple band pass filters, each of which is associated with a specific band of audio frequencies—for example, a digital filter bank having 12 digital Butterworth band pass filters of 6th order, Infinite Impulse Response (IIR) type—so that the acoustic audio signal is filtered into some M band pass signals, B1 to BM where each signal corresponds to the band of frequencies for one of the band pass filters. Each output of the CIS band pass filters can roughly be regarded as a sinusoid at the center frequency of the band pass filter which is modulated by the envelope signal. This is due to the quality factor (Q≈3) of the filters. In case of a voiced speech segment, this envelope is approximately periodic, and the repetition rate is equal to the pitch frequency. Alternatively and without limitation, the Preprocessor Filter Bank 201 may be implemented based on use of a fast Fourier transform (FFT) or a short-time Fourier transform (STFT). Based on the tonotopic organization of the cochlea, each stimulation contact in the scala tympani often is associated with a specific band pass filter of the external filter bank.
FIG. 3 shows an example of a short time period of an audio speech signal from a microphone, and FIG. 4 shows an acoustic microphone signal decomposed by band-pass filtering by a bank of filters into a set of signals. An example of pseudocode for an infinite impulse response (IIR) filter bank based on a direct form II transposed structure is given by Fontaine et al., Brian Hears: Online Auditory Processing Using Vectorization Over Channels, Frontiers in Neuroinformatics, 2011; incorporated herein by reference in its entirety:
for j = 0 to number of channels − 1 do for s = 0 to number of samples − 1 do  Yj(s) = B0j * Xj (s) + Z0j  for i = 0 to order − 3 do   Zi,j = Bi+1,j * Xj(s) + Zi+1,j − Ai+1,j * Yj(s)  end for  Zorder − 2,j = Border − 1,j * Xj(s) − Aorder − 1,j * Yj(s) end forend for
The band pass signals B1 to BM (which can also be thought of as frequency channels) are input to a Signal Processor 202 which extracts signal specific stimulation information—e.g., envelope information, phase information, timing of requested stimulation events, etc.—into a set of N stimulation channel signals S1 to SN that represent electrode specific requested stimulation events. For example, channel specific sampling sequences (CSSS) may be used as described in U.S. Pat. No. 6,594,525, which is incorporated herein by reference in its entirety. For example, the envelope extraction may be performed using 12 rectifiers and 12 digital Butterworth low pass filters of 2nd order, IIR-type.
A Pulse Generator 205 includes a Pulse Mapping Module 203 that applies a nonlinear mapping function (typically logarithmic) to the amplitude of each band-pass envelope. This mapping function—for example, using instantaneous nonlinear compression of the envelope signal (map law)—typically is adapted to the needs of the individual cochlear implant user during fitting of the implant in order to achieve natural loudness growth. This may be in the specific form of functions that are applied to each requested stimulation event signal S1 to SN that reflect patient-specific perceptual characteristics to produce a set of electrode stimulation signals A1 to AM that provide an optimal electric representation of the acoustic signal. A logarithmic function with a form-factor C typically may be applied as a loudness mapping function, which typically is identical across all the band pass analysis channels. In different systems, different specific loudness mapping functions other than a logarithmic function may be used, with just one identical function is applied to all channels or one individual function for each channel to produce the electrode stimulation signals A1 to AM outputs from the Pulse Mapping Module 203.
The Pulse Generator 205 also includes a Pulse Shaper 204 that develops the set of electrode stimulation signals A1 to AM into a set of output electrode pulses E1 to EM for the electrode contacts in the implanted electrode array which stimulate the adjacent nerve tissue. The electrode stimulation signals A1 to AM may be symmetrical biphasic current pulses with amplitudes that are directly obtained from the compressed envelope signals.
In the specific case of a CIS system, the stimulation pulses are applied in a strictly non-overlapping sequence. Thus, as a typical CIS-feature, only one electrode channel is active at a time and the overall stimulation rate is comparatively high. For example, assuming an overall stimulation rate of 18 kpps and a 12 channel filter bank, the stimulation rate per channel is 1.5 kpps. Such a stimulation rate per channel usually is sufficient for adequate temporal representation of the envelope signal. The maximum overall stimulation rate is limited by the minimum phase duration per pulse. The phase duration cannot be arbitrarily short because, the shorter the pulses, the higher the current amplitudes have to be to elicit action potentials in neurons, and current amplitudes are limited for various practical reasons. For an overall stimulation rate of 18 kpps, the phase duration is 27 μs, which is near the lower limit.
In the CIS strategy, the signal processor only uses the band pass signal envelopes for further processing, i.e., they contain the entire stimulation information. For each electrode channel, the signal envelope is represented as a sequence of biphasic pulses at a constant repetition rate. A characteristic feature of CIS is that the stimulation rate is equal for all electrode channels and there is no relation to the center frequencies of the individual channels. It is intended that the pulse repetition rate is not a temporal cue for the patient (i.e., it should be sufficiently high so that the patient does not perceive tones with a frequency equal to the pulse repetition rate). The pulse repetition rate is usually chosen at greater than twice the bandwidth of the envelope signals (based on the Nyquist theorem).
Another cochlear implant stimulation strategy that does transmit fine time structure information is the Fine Structure Processing (FSP) strategy by Med-El. Zero crossings of the band pass filtered time signals are tracked, and at each negative to positive zero crossing, a Channel Specific Sampling Sequence (CSSS) is started. Typically CSSS sequences are only applied on the first one or two most apical electrode channels, covering the frequency range up to 200 or 330 Hz. The FSP arrangement is described further in Hochmair I, Nopp P, Jolly C, Schmidt M, Schöβer H, Garnham C, Anderson I, MED-EL Cochlear Implants: State of the Art and a Glimpse into the Future, Trends in Amplification, vol. 10, 201-219, 2006, which is incorporated herein by reference.
Many cochlear implant coding strategies use what is referred to as an N-of-M approach where only some number n electrode channels with the greatest amplitude are stimulated in a given sampling time frame. If, for a given time frame, the amplitude of a specific electrode channel remains higher than the amplitudes of other channels, then that channel will be selected for the whole time frame. Subsequently, the number of electrode channels that are available for coding information is reduced by one, which results in a clustering of stimulation pulses. Thus, fewer electrode channels are available for coding important temporal and spectral properties of the sound signal such as speech onset.
One method to reduce the spectral clustering of stimulation per time frame is the MP3000™ coding strategy by Cochlear Ltd, which uses a spectral masking model on the electrode channels. Another method that inherently enhances coding of speech onsets is the ClearVoice™ coding strategy used by Advanced Bionics Corp, which selects electrode channels having a high signal to noise ratio. U.S. Patent Publication 2005/0203589 (which is incorporated herein by reference in its entirety) describes how to organize electrode channels into two or more groups per time frame. The decision which electrode channels to select is based on the amplitude of the signal envelopes.
In addition to the specific processing and coding approaches discussed above, different specific pulse stimulation modes are possible to deliver the stimulation pulses with specific electrodes—i.e. mono-polar, bi-polar, tri-polar, multi-polar, and phased-array stimulation. And there also are different stimulation pulse shapes—i.e. biphasic, symmetric triphasic, asymmetric triphasic pulses, or asymmetric pulse shapes. These various pulse stimulation modes and pulse shapes each provide different benefits; for example, higher tonotopic selectivity, smaller electrical thresholds, higher electric dynamic range, less unwanted side-effects such as facial nerve stimulation, etc. But some stimulation arrangements are quite power consuming, especially when neighboring electrodes are used as current sinks. Up to 10 dB more charge might be required than with simple mono-polar stimulation concepts (if the power-consuming pulse shapes or stimulation modes are used continuously).
It is well-known in the field that electric stimulation at different locations within the cochlea produce different frequency percepts. The underlying mechanism in normal acoustic hearing is referred to as the tonotopic principle. In cochlear implant users, the tonotopic organization of the cochlea has been investigated (e.g. by Vermeire et al., Neural tonotopy in cochlear implants: An evaluation in unilateral cochlear implant patients with unilateral deafness and tinnitus, Hear Res, 245(1-2), 2008 Sep. 12 p. 98-106; Schatzer et al., Electric-acoustic pitch comparisons in single-sided-deaf cochlear implant users: Frequency-place functions and rate pitch, Hear Res, 309, 2014 March, p. 26-35; both of which are incorporated herein by reference in their entireties). High rate single electrode stimuli have been matched to acoustic stimuli received through a normal hearing contralateral ear. FIG. 5 shows an example of prototype frequency percept in response to high rate stimulation (1500 pps) over distance along the basilar membrane from the round window.
It is also known in the field that different stimulation rates produce different pitch percepts at different apical electrodes (Schatzer et al., 2014; Prentiss et al., Ipsilateral acoustic electric pitch matching: a case study of cochlear implantation in an up-sloping hearing loss with preserved hearing across multiple frequencies, Cochlear Implants Int., 15(3), 2014 May, p. 161-165; incorporated herein by reference in its entirety). More apical electrodes produce even lower pitch percepts at lower stimulation rates. On the other hand, rate-pitch saturation limits (i.e. the rate above which no more useful change in pitch occurs) have been identified to be as high as 900 Hz (Kong et al., Temporal pitch perception at high rates in cochlear implants, J. Acoust. Soc. Am., 127(5), 2010 May, p. 3114-3123; incorporated herein by reference in its entirety). On individual electrodes, preferably in the apical region of the cochlea, the frequency range that is perceivable by varying the stimulation rate can be relatively large.
FIG. 6 shows examples of the frequency percepts elicited at different intracochlear stimulation locations by different stimulation rates. On an apical electrode, a stimulation rate of 100 pps might sound like an acoustic sound with a fundamental frequency (F0) of 80 Hz, whereas a stimulation rate of 1500 pps might sound like an acoustic stimulus with an F0 of 900 Hz. However, the perceived frequency for a given stimulation location and rate can also change as a function of stimulation level (or current in μA; Vandali et al., Pitch and loudness matching of unmodulated and modulated stimuli in cochlear implantees, Hear Res., 302, 2013 August, p. 32-49, incorporated herein by reference in its entirety).
In a typical cochlear implant system, the most apical electrode is assigned to a relatively narrow band pass filter, e.g. 100-200 Hz. In coding strategies such as FSP, which are designed to code temporal fine structure information, typical stimulation rates of 100 to 200 pps are derived for the specific electrode. Such systems are known to have various limitations including:                A possible change of perceived frequency as a function of stimulation current level for a given stimulation location and rate is not taken into account (see FIG. 7 which shows frequency percepts elicited at different intracochlear stimulation locations by different stimulation rates delivered at different stimulation current levels).        The available range of perceived frequencies coded by different stimulation rates is restricted to the band pass limits of the assigned filters (see FIG. 8 which shows non-overlapping frequency ranges perceived by stimulating three adjacent electrode locations at band-specific stimulation rates).        A change in instantaneous frequency of an input frequency component is only partly translated into a change of stimulation location (Instantaneous stimulation rates within the analyzed frequency range stay at the location of the electrode, see FIG. 8).        Multiple frequency components within one band pass filter cannot be assigned a specific stimulation location and rate.        
Band pass filters define the frequency range transmitted by a given physical electrode, or by a virtual electrode that stimulates two adjacent electrodes either simultaneously or in rapid succession. Temporal fine structure coding strategies apply rate code to an electrode assigned to a given filter band. Adjacent electrodes or electrode combinations are typically assigned to adjacent filter bands. Individual frequency components can be assigned to pre-defined, amplitude-weighted simultaneously-stimulated electrodes, as in U.S. Patent Publication 2010/0185261 (which is incorporated herein by reference in its entirety). This type of signal analysis limits electrode pitches to the assigned frequency range and disregards lower and higher pitches produced by rates that are lower or higher, respectively, than the band frequency limits. In addition, continuous changes in input frequency are only represented as continuous changes in stimulation rate or rate of pulse packages, but not as continuous shifts in simulation location. And also, the influence on perceived frequency of stimulation level for a given stimulation location and rate is not taken into account.
U.S. Pat. No. 8,554,330 describes a different method of electrode location-pitch matching by indirectly measuring an individual cochlear location-frequency map based on acoustic auditory brainstem response (ABR). This individual location-frequency map is used to either position an electrode within the cochlea so as to match the electrode location to the individual location-frequency map, or to map an already inserted electrode so as to provide “stimuli only to the parts of the cochlea that have reduced or no residual hearing.” This approach requires that residual hearing be sufficiently preserved to measure the individual location-frequency map via acoustic ABR, and therefore is not usable for cochlear implant patients without residual acoustic hearing. In addition, the pitch of electrodes is matched to the individual location-frequency map solely by manipulating the stimulation location, which in yet-to-be-implanted patients is the desired insertion depth of the electrode, or in already implanted electrodes is the electrode(s) to be activated.
U.S. Pat. No. 8,532,782 (Musical Fitting of Cochlear Implants; incorporated herein by reference in its entirety) describes a method for deriving electrode weighting factors for simultaneous electrode stimulation, i.e. for matching virtual-channel stimulation location to musical-interval rate-pitch percepts in a cochlear implant. But, the described method does not take into account individual rate-pitch saturation functions and uses a 1:1 mapping of the acoustic component frequencies to the electrical pulse rates.
U.S. Pat. No. 7,979,135 (Cochlear Implant Pitch Intensity; incorporated herein by reference in its entirety) describes a method for generating “electrode stimulation signals which have the intensity levels that reflect the pitch characteristics” of the acoustic stimulus frequency components, based upon “the relationship between stimulus intensity and pitch perception” in cochlear implants.